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C omputational science and engi- 
neering (CS&E) describes a 
researcher’s use of computers to 
simulate physical processes. CS&E paral- 
lels the development of the two other 
modes of science: theoretical and experi- 
mental/observational. 

In addition to new methodologies, new 
technologies or mathematical tools have 
spurred the scientific revolutions. For 
example, calculus allowed Newton to 
codify the laws of nature mathematically 
and develop analytic methods for solving 
simple cases. Similarly, the development 
of the von Neumann computer architecture 
gave scientists the ability to solve the dis- 
cretized laws of nature for general and 
complex cases. 

CS&E now relies heavily on scientific 
visualization to represent these solutions, 
enabling scientists to turn mountains of 
numbers into movies and graphically dis- 
play measurements of physical variables 
in space and time. This article explores the 
convergence of science and visualization, 
in support of its successful growth and 
development. 


What is scientific 
visualization? 

Computer graphics and image process- 
ing are technologies. Visualization, a term 


used in the industry since the 1987 publica- 
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Visualization holds 
great promise for 
computational science 
and engineering, 
provided we can meet 
the immediate and 
long-term needs of 
both toolmakers and 
tool users. 





tion of the Nationa! Science Foundation 
report Visualization in Scientific Comput- 
ing,' represents much more than that. Visu- 
alization is a form of communication that 
transcends application and technological 


boundaries. 


A tool for discovery and understand- 
ing. The deluge of data generated by super- 
computers and other high-volume data 
sources (such as medical imaging systems 
and satellites) makes it impossible for 
users to quantitatively examine more than 
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a tiny fraction of a given solution. That is, 
it is impossible to investigate the qualita- 
tive global nature of numerical solutions. 

With the advent of raster graphics, re- 
searchers can convert entire fields of vari- 
ables (representing density, pressure, ve- 
locity, entropy, and so on) to color images. 
The information conveyed to the re- 
searcher undergoes a qualitative change 
because it brings the eye-brain system, 
with its great pattern-recognition capabili- 
ties, into play in a way that is impossible 
with purely numeric data. 

For example, an observer instantly sees 
the vortices, shock systems, and flow pat- 
terns in a visualization of a hydrodynamic 
calculation, while these same patterns are 
invisible in mere listings of several 
hundred thousand numbers, each repre- 
senting field quantities at one moment in 
time. When computing a space-time solu- 
tion to the laws of physics, the particular 
numeric quantities at each event in time- 
space are not important; rather, what is 
important is understanding the global 
structure of the field variables that consti- 
tute the solution and the causal intercon- 
nections of the various components of that 
solution. 


A tool for communication and teach- 
ing. Much of modern science can no longer 
be communicated in print. DNA se- 
quences, molecular models, medical imag- 
ing scans, brain maps, simulated flights 
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through a terrain, simulations of fluid flow, 
and so on, all need to be expressed and 
taught visually over time. To understand, 
discover, or communicate phenomena, 
scientists want to compute the phenomena 
over time, create a series of images that 
illustrate the interrelationships of various 
parameters at specific time periods, down- 
load these images to local workstations for 
analysis, and record and play back one or 
more seconds of the animation. 

According to the visualization report, 
“We speak (and hear) — and for 5000 
years have preserved our words. But, we 
cannot share vision. To this oversight of 
evolution we owe the retardation of visual 
communication compared to language. 
Visualization by shared communication 
would be much easier if each of us had a 
CRT in the forehead.”' 

Our CRTs, although not implanted in 
our foreheads, are connected to computers 
that are nothing more than extensions of 
our brains. These computers, however, 
might not be in the same room with us. 
They could be down the hall, across town, 
or across the country. Hence, the ability to 
communicate visually — and remotely — 
with computers and each other depends on 
the accessibility, affordability, and per- 
formance of computers and computer net- 
works. | 


The visualization report recommends 
the development of a federally funded ini- 
tiative providing immediate and long-term 
funding of both research and technology 
developments (see Table 1).' Research 
developments are the responsibility of tool 
users — experts from engineering and the 
discipline sciences who depend on compu- 
tations for their research. Technology 
developments are handled by toolmakers 
— the visualization researchers who can 
develop the necessary hardware, software, 
and systems. 


Tool users’ short-term | 
needs 


Every researcher requires a personal 
computer or workstation on his or her desk 
connected with a remote supercomputer. 
However, not all scientists require the 
same level of computing power. Hence, a 
three-tiered model environment is begin- 
ning to emerge that categorizes visualiza- 
tion systems by such factors as power, 
cost, and software support. 


Table 1. Recommendations for a national initiative on visualization in scientific 


computing. 








engineers 


Toolmakers: 

Visualization 
scientists and 
engineers 


Table 2. Visualization facility three-tiered hierarchy. 


Model A 


Hardware 


Bandwidth (potential >10° 
interactive rates, 
bits/second) 


Location (where users 
interact with the 
display screen) 


Software (in addition 
to discipline-specific 
_ data generation and 


processing) 
capabilities 


Administration 
Strength: 


Weakness: 
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Supercomputer or 
super image computer 


Machine room 
(at the center) 


Commercial packages for 
output only (no steering). 
Research required to 
develop interactive steering 


Support staff 


Centralization 


Minisupercomputer or 
image computer 


10’-10° 


local area network 


Commercial packages 
interaction is becoming 
to improve discipline- 


specific interaction 


Discipline-specific 
visualization goals 


Small support staff 


Tool users: Funding to incorporate 
Computational _ visualization in 
scientists and current research 


No funding necessary 


Laboratory on a high-speed 


available. Research required 


| Short-term Needs Long-term Needs | 


Funding to use model 
visualization environments 


Funding to develop model 
visualization environments 


Model C 


Advanced workstations (mini-/ 
micro- image computer 


10°-10° 


Laboratory on a national/ 
regional network 


Commercial packages and 


are mostly output only. Some tools are widely available 


for both computation and 

interaction. Research required 
. in languages, operating 

systems, and networking 


Decentralization 


No support staff — 
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Workstations. Researchers need work- 
stations with access to supercomputers for 


e immediate access to local graphics 
capabilities, 
’ networked access to supercomputers, 
and 
e hard-copy recording. 


Local graphics. Workstations, mini- 
computers, and image computers are sig- 
nificantly more affordable than supercom- 
puters, and they are more powerful and 
effective visualization tools. There are 
already some 20 million personal comput- 
ers and workstations in the United States, 
compared with about 200 supercomputers. 
Workstation users are increasingly treat- 
ing supercomputers as one of many win- 


dows on the screen, and scientists must be 
able to “cut and paste” between the super- 
computer and applications running on their 
local machines. 


Access to supercomputers. Scientists 
need to transfer data to and from a main 
computation device, but today’s networks 
are too slow for use in visualization. Some 
temporary techniques reduce the demand 
for high bandwidth, such as off-peak im- 
age transmission, image compression, 


image reconstruction from abstract repre-_ 


sentations, and local image generation. 
Networking is therefore as critical as 
computer power in helping scientists. 


Hard-copy recording. Whether the visu- 


als are for personal analysis, information 
sharing among peers, or presentations in 
formal surroundings, equipment for pro- 
ducing photographs, slides, videotapes, or 
laser disks needs to be in place and as easy 
to use as sending text files to a laser printer. 

Scientists need the ability to create ad 
hoc graphics to verify the integrity of their 
simulations, gain insights from their analy- 
ses, and communicate their findings to 
others. Low-cost animation facilities 
should be connected to every user worksta- 
tion so researchers can make scientific 
“home movies” with little effort. High-end 
visualization capabilities and facilities 
also should be available at all research 
centers; high-end graphics become impor- 
tant for presentation and publication of 
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Low-cost, visualization-compatible 


Our ability to communicate visually and remotely with 
supercomputers and each other depends on 


(1) the ease with which we can use our office/fhome com- 
puters to connect with the outside world, receive and transmit 
visual information, and record this information on videotapes 
or slides, and 

(2) the cost/performance of today’s networks. 


The Electronic Visualization Laboratory (EVL) at the Uni- 
versity of Illinois at Chicago is doing research in both areas. 
We are designing as our scientific animation workstation a 
low-cost computer system with a well-integrated visualization 
programming environment. 

Users at the National Center for Supercomputing Applica- 
tions (NCSA) — or any of the National Science Foundation- 
funded supercomputer centers, for that matter —- cannot do 
graphics remotely due to slow network speeds, centralized 
and expensive graphics equipment, lack of graphics software 
tools, and the need for specialists in film/video production. 
Our research is motivated by the recent availability of low- 
cost graphics hardware and a good PC-based visualization 
toolkit, coupled with a growing awareness that scientists 
need visualization more for personal/peer analysis than for 
presentations. 

EVL is integrating affordable commercial equipment with 
specially designed graphics software to make visualization a 


- reality for computational scientists — whether they use their 


computers on a stand-alone basis or connected to supercom- 
puters over networks. (Regarding affordability, academicians 
can generally receive $10,000 in equipment monies from their 
departments or colleges without applying for external grants 
— our yardstick is that equipment should cost no more than a 
three-year-old Buick.) 

EVL's scientific animation workstation, shown in the ac- 
companying figure, has hard-copy recording capability and 
an easy-to-use visualization environment to facilitate scien- 
tists’ needs. The following list corresponds to items 1-6 in 
the figure. 


(1) Supercomputer access. Supercomputers are most ef- 


workstations and networks 


ficiently used to run complex simulation codes, the output of 
which is numbers. With access to graphics, researchers can 
convert numbers to pictures to qualitatively examine the global 
nature of their simulation output. Graphics can be made avail- 
able on the host machine or, more efficiently, on the local work- 
station. 


(2) Televisualization: graphical networking. As images re- 


quire more colors, higher resolution, or larger volumes of data, 


they need more memory and become more impractical to trans- 
mit over networks or phone lines, to store on disks. or to convert 
and display on different frame buffers. EVL’s {mcomp compres- 
sion and conversion software converts images consisting of 24, 
16, or 8 bits per pixel to 16 or 8 bits per pixel, then compresses 
them further to 2 or 3 bits per pixel while maintaining a reason- 


able full-color representation.'* The program takes only 0.4 sec- 


onds to run on the Cray X-MP at NCSA, and it converts and 
transmits a 512 x 512 x 24-bit image from NCSA to EVL over a 
56-kilobyte line within a few seconds. 

Moreover, visuals must be transmitted from memory to mem- 
ory (that is, from supercomputer memory to frame buffer mem- 
ory in the local computer), not just from file to file as in elec- 
tronic mail-type networks. NCSA's Teinet communications soft- 
ware has been modified to do this and expanded to include 
Imcomp routines that automatically compress images. 

in addition to compression, value-added nodes speed up gra- 
phical transmission by balancing transmission costs with local 
computing costs. Model data is sent over networks and then 
rendered or reconstructed at the scientist's end. EVL is cur- 
rently investigating the use of its AT&T Pixel Machine as a 
graphics server that would render model data transmitted over 
the network from the supercomputer and then transmit the re- 
sulting images over a local area network to individuals’ desktop 
computers. . 


(3) Truevision Vista graphics board. Scientists need to be 
able to preview, record, and play back animations at any speed 
and in cyclical fashion to examine the dynamics of their data 
changing over time, to spot anomalies, or to uncover computa- 
tion errors. The Vista board’s large configurable memory aliows 
us to get anywhere from 32 screens at 512 x 512 pixels to 128 
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results once researchers conclude their 
work. 


Three-tiered model computational 
environment. Observations of the way 
scientists use visualization suggest that a 
three-tiered model environment is evolv- 
ing, as defined in Table 2. Each model is 


distinguished by hardware costs, comput- | 


ing power, bandwidth, location, software 
support, and administrative considera- 
tions.! 

This model environment assumes that 
scientists want as direct a visual connec- 
tion to their computations as possible. 
While supercomputers (model A) provide 
scientists with powerful number-crunch- 
ing tools for generating data, they cur- 


rently do not produce graphics; they do fill 


arrays with information that somehow gets 
piped to display devices. (Table 2 assumes 
that supercomputers and super image 
computers have equivalent power. Super 
image computers, although not commer- 
cially available today except in the form of 
a special-purpose flight simulator, will 
provide the specialized processing neces- 
sary for real-time volume visualization.) 
Workstations give scientists more con- 
trol over their visual output (models B and 
C). A workstation typically addresses its 
display memory the same way it addresses 
regular memory, incurring essentially no 
hardware overhead to display computed 


results. (Table 2 also assumes that minisu- | 


percomputers and image computers have 

















equivalent power, and that advanced work- 
stations and mini-/micro- image comput- 
ers have equivalent power.) . 
Scientists should be able to select either 
more-expensive workstations with power- 
ful visualization potential (model B) or 
less expensive ones (model C) while main- 
taining network connections to larger 
machines (model A) to do computations 
when necessary. This interdependency can 
work quite well. For example, 2 scientist 
can calculate 20-60 frames of a simulation 
sequence on a supercomputer, download 
the images to a workstation to create a 
minimovie, and then play back the se- 
quence at any speed under local control.?? 
(See sidebar, “Low-cost, visualization- 
compatible workstations and networks.”) 





Televisualization: vNonaiiag | 
graphical : a 
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board programming environment 





The Electronic Visualization Laboratory’s RT/1 graphics language, an 80386-based personal computer, the Truevision 
Vista board, and consumer video gear comprise a scientific animation production facility that is economical enough to 
be made available to research scientists and engineers on a broad scale. 


screens at 128 x 128 pixels, all at 8 bits per pixel. The board is 
also video compatible, so images can be recorded directly to 


videotape. 


(4) Real Time/One (RT/1) loca! visualization programming 


and record frames of animation.? This equipment also comes 
with a built-in microphone so scientists can add narration or 


other sounds to visual recordings. 


(6) Color monitor. Today's consumer video systems not 





environment. Scientists need a set of tools for picture composi- 
tion, picture saving/restoring, fonts and text, resizing, rotation, 
moving, copying, hand retouching (painting), color manipulation, 
etc. They also need a local graphics programming environment 
in which to develop new tools or extend the capabilities of exist- 
ing ones. 

RT/1, an easy-to-use graphics programming language devei- 
oped by EVL faculty and students, meets the criteria required of 
a visualization system environment. The language, written in C 
and running under Unix and MS-DOS, runs on all of EVL’s 
workstations and personal computers. EVL is porting RT/1 to 
new workstations as they are acquired, extending the capabili- 
ties of the language, and developing application programs 
tailored to the needs of scientists. 


(5) Consumer video recorder/player. if it's not recordable, 
it’s not science. Moreover, the equipment for producing video- 
tapes needs to be as easy to use as sending text files to a laser 
printer. We are integrating low-cost consumer video equipment 
into the workstation so scientists can quickly and easily preview 


August 1989 


only record but also can be attached to any television for im- 
mediate viewing of recorded material. Scientists can take a 
small video unit to a conference and plug it into a television 
there to share findings with colleagues. Should peers in other 
towns have similar equipment, colleagues could mail tapes to 
each other for viewing. 
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Table 3. Total corporate computing needs. (Source: Larry Smarr, NCSA, Sept. 1988.) 


Computational Science and Engineering Data Processing 


Corporate officer 
responsible 


Tiered architectures 


Vice president of research 
or long-range planning 


Personal computers and graphics 
workstations; midrange machines 
(mainframes/minisupercomputers; 
supercomputers 

Need exists for multivendor, networked, 


hierarchical computing. 


Open systems 

No vendor has emerged who offers integrated 
systems and end-to-end solutions. As a result, 
end users are faced with a confusing set of 


products from various vendors and nowhere to 
turn for advice on how to integrate them. 


Vendors 


Fragmented market populated by start-ups 
and extremely high-growth companies: 


Workstations (Sun, DEC, Apollo, IBM, 
Hewlett-Packard, Apple, Silicon Graphics, 
Ardent, Stellar, AT&T Pixel, etc.); Midrange 
(DEC, IBM, Alliant, Amdahl, Convex, 
Scientific Computing Systems, Multiflow, 
Elxsi); Supercomputers (Cray, IBM). 


Operating Systems Unix 
Networking protocols; 
telecommunications; 
speeds 


Open network standards; 

Long-haul telecommunications; 
High speed = 1,000 Mbits/second 
Because of the scarcity of $20 million 
supercomputers, most universities and 
corporate CS&E users are remote and 
must gain access to supercomputers over 
long-haul telecommunication lines. 


Image (megabyte) 


Vice president of management 
information systems (MIS) 


Personal computers; minicomputers; 
mainframes —& 

Software portability only partially exists between 
these levels, and then only within one vendor's 
product line. 


Closed systems 

IBM and Digital Equipment Corporation 
manufacture all levels of computers and the 
connections between them. 


Mature, slow-growth marketplace dominated 
by a few giant vendors, such as IBM and DEC. 


MVS, DOS, VMS (proprietary) 


SNA, DECnet (proprietary); 

High speed = 50 Mbits/second 

Within a corporation, most networks hook many 
“dumb terminals” up to a central mainframe where 
all the computing power resides. PCs are 

generally used stand-alone; those networked to a 
mainframe generally use the network to download 
or upload files, and computing is decoupled. 


Common unit Number (byte) 
of information Supercomputer simulations produce such 
enormous amounts of data that visualization is a 
essential. BS 
Common unit Mflops MIPS 


for computation speed 


Additional models D, E, and F, corre- 
sponding to personal computers, alphanu- 
meric CRT terminals, and batch output, 
respectively, also exist. They do not repre- 
sent advanced visualization technology, so 
they are not included in-our model environ- 
ment. Note, however, that model F has 
been used to produce a great deal of anima- 
tion for both the scientific and commercial 
entertainment industries for the past 20 
years. 
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Tool users’ long-term 
needs 


CS&E is emerging as a new marketplace 
with needs distinct from those of data 
processing, as shown in Table 3. Success 
in the CS&E marketplace of the 1990s will 
depend on acommitment to standards, ease 
of use, connectivity, open systems, inte- 
grated systems, software portability, multi- 


vendor environments, leading-edge tech- 
nology, and customer service and support.‘ 

The list of research opportunities for 
visualization in scientific computing is 
long and spans all of contemporary scien- 
tific endeavor. The sidebar “Scientific and 
engineering research opportunities” pre- 
sents specific examples of advanced scien- 
tific and engineering applications to show 


(Continued on p. 22) 
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Molecular modeling. The use of inter- 
active computer graphics to gain insight 
into chemical complexity began in 1964. 
Interactive graphics is now an integral 
part of academic and industrial research 
on molecular structures, and the method- 
ology is being successfully combined with 


supercomputers to model complex sys- 
tems. Two types of images can currently 
be generated: realistic pictures of mole- 
cules and 3D line drawings. Raster 
equipment is used to create realistic rep- 
resentations and animations, while vector 
hardware, used for real-time display and 
interaction, creates line drawings. 

The image at left is a 3D line drawing 
of the rhinovirus, the common cold virus, 
showing its geometric structure and com- 
plexity. The image at right is an artistic 
rendering of the human papiiloma virus 
(HPV). It was done by a group of Chi- 
cago-area artists who appreciate the 


underlying mathematics of nature and the - 


complexity of the inner workings between 
atoms. ' 


Left-hand © 1988 T.J. O’Donnell. Data courtesy of 
Dr. Rossman, Crystallography Group. Purdue Univ. 
Image courtesy of the EVL, Univ. of fllinois at Chi- 
cago. Right-hand © 1989 (Art)" Laboratory, Illinois 
Institute of Technology. (Art)" artists: Donna Cox, 
NCSA, Univ. of Illinois at Urbana-Champaign: 
Stephan Meyers, Dan Sandin, and Tom DeFanti, EVL, 
Univ. of Illinois at Chicago; Ellen Sandor. (Art)" 
Laboratory, [Illinois Institute of Technology. 








Medical imaging. Scientific computa- 
tion applied to medical imaging has cre- 
ated opportunities in diagnostic medicine, 
surgical planning for orthopedic prosthe- 
ses, and radiation treatment planning. In 
each case, these opportunities have been 
brought about by 2D and 3D visualiza- 
tions of portions of the body previously 
inaccessible to view. © 

The above-left image is a shaded sur- 
face volume rendering of a 128 x 128 x 
197 computerized tomography scan of a 
tree sloth. The opacity of various struc- 
tures can be interactively modified to 
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show the skin surface or to reveal inter- 
nal structures. The bones of the rib cage, 
shoulder blades, and spine can be seen 
in the image on the right, as well as the 
trachea, lungs, heart and diaphragm. 
The above-right image is a shaded sur- 
face volume rendering of a 256 x 256 x 
61 magnetic-resonance imagery (MRI) 
scan of a human head. The rendering 
shows a mixture of surface and slice- 
based techniques, where external struc- 
tures such as the skin are rendered with 
surface shading, while slice planes are 
voxel-mapped to reveal! the original MRI 


data. Physicians can use this technique 
to relate the position of internal structures 
such as tumor sites to external land- 
marks. These images were generated us- 
ing the Voxvu volume rendering tool on a 
Sun workstation with the TAAC-1 Appli- 
cation Accelerator. 


© 1989 Chuck Mosher and Ruth Johnson, Sun Mi- 
crosystems. Data for above-left image courtesy of Eric 
Hoffman, UPA. Data for above-right image courtesy of 
Jeff Shaw, Vanderbilt Univ. 
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Brain structure and function. Rutgers 
University is using computer vision and 
visualization methods to automatically 
detect white-matter lesions in MR! scans 
of the human brain. 

In the above-left image, low-level vision 
methods locate the outline of the brain, 
landmarks such as the interhemispherical 
fissure plane, and suspected lesions. The 


system calculates the orientation of the 
brain and uses the segmentations pro- 
vided by the low-level methods to fit a de- 
formable model to each patient's brain to 
determine the position and shape of diffi- 
Cult-to-identify organs or regions of inter- 
est. This customized model, shown in the 
above-right image, is used to obtain infor- 
mation about the anatomical position of 





the suspected lesions so that the system 
can reject false positives and determine 
the affected organs. The system has 
been tested on more than 1,200 images 
from 19 patients, producing good results. 


© 1989 Ioannis Kapouleas, Computer Science Dept., 
Rutgers Univ. 
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Mathematics. These images illustrate 
a type of fractal known as the Julia set. A 
filled-in Julia set is a set of points that do 
not converge (or diverge) to infinity after 
repeated applications of a function, such 
as f(z) = z? + c. These functions are of- 
ten investigated in the complex plane, but 
they also exist in the quaternions, a coor- 
dinate system that spans one real and 
three imaginary axes. Visualization helps 


‘ mathematicians understand these equa- 


tions, which are too complex to conceptu- 
alize otherwise.” 


The above-left image is a quaternion 
filied-in Julia set minus its front-upper-left 
octant; the inner components of the four- 
cycie are revealed, defining its basin of 
attraction. The above-right image is a 
visualization of a dendritic quaternion Ju- 
lia set in the complex plane; the unusual 
lighting uses a 3D gradient in the com- 
plex plane. 


© 1989 John Hart, EVL, Univ. of Illinois at Chicago. 
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Geosciences: meteorology. The 
study of severe storms through observa- 
tion and modeling helps research meteor- 
ologists understand the atmospheric con- 
ditions that breed large and violent torna- 
does and the mechanisms by which tor- 
nadoes form and persist.? Theoreticians 
and field workers obtain information on 
behavior that cannot be safely observed; 
study the interactions of various environ- 
ments, characterized by differing vertical 
wind, temperature, pressure, and mois- 
ture structures; and obtain useful guides 
for future research. 

Transparency and volumetric rendering 





Space exploration. The field of piane- 
tary study involves the accumulation of 
huge volumes of data on the planets in 
the solar system. Enough data is now 
available that scientists are beginning to 
integrate observed phenomena and the- 
ory from other fields involved in planetary 
study: meteorology, geography, planetary 
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are used to view multiple surfaces; shad- 
ing is used to display individual solid sur- 
faces. | 

The above-left image uses voxel (grid 
cell) data to display rainwater and vertical 
vorticity information about a storm,’ pro- 
viding scientists with more information 
than if they had observed the storm with 
their eyes. The fuzzy region indicates low 
rainwater amounts while the bright white 
regions indicate large amounts of rain- 
water within the cloud. The vertical vor- 
ticity is texture mapped onto the rain- 
water with color; purple indicates domi- 
nant positive vorticity and blue indicates 


physics, astronomy, and astrophysics. 
The above-left image is from an ani- 
mated simulation of the dynamics of Ura- 
nus’ magnetosphere. The simulation 
shows that the angle of the dipole axis 
(purple arrow) is offset from the planet's 
angle of rotation (aqua arrow). The 
above-right image is from a simulation of 


dominant negative vorticity. 

The above-right image is from an ani- 
mated simulation of a storm over Kansas, 
in which the rainwater surface was poly- 
gonized (tiled) and then rendered. The 
simulation clearly reveals substantial vari- 
ations in the structure of the rainwater 
field not apparent earlier. 


Above-left © 1988 Robert Wilthelmson and Craig 
Upson, NCSA, Univ. of Iflinois at Urbana-Champaign. 
Above-right © 1988 Robert Wilhelmson. Crystal 
Shaw. Lou Wicker. Stefen Fangmeier, and the NCSA 
Visualization Production Team. Univ. of Illinois at 
Urbana-Champaign. 





the Voyager 2 Neptune encounter to oc- 
cur in jate summer of 1989. This image il- 
lustrates the path of the Voyager 2 as 
viewed from Earth. 


Above-left © 1989 Computer Graphics Group of the 
Jet Propulsion Laboratory and G. Hannes Voigt of Rice 
Univ. Above-right © 1989 Computer Graphics Group 
of the Jet Propulsion Laboratory. 
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Astrophysics. Computational astro- 
physicists at the NCSA work with artists 
in an attempt to see the unseen and cre- 
ate visual paradigms for phenomena that 
have no known visual representation. 

An embedding diagram of a Schwarz- 
schild black hole and the behavior of its 
gravitational field, illustrated in the above- 
left image, was obtained from a numeri- 
cal solution of Einstein’s numerical rela- 
tivity equations. The surface of the dia- 
gram measures the curvature of space 


due to the presence of the black hole, 
while the color scale represents the 
speed at which idealized clocks measure 
time (with red representing the slowest 


clocks and biue representing the fastest). 


A black hole emits gravitational radia- 
tion after it has been struck by an incom- 
ing gravity wave. The above-right image 
is from an animated sequence that 
shows, for the first time, the influence of 
the curved space on the propagation of 
the radiation. Through the use of an iso- 


‘a 


metric embedding diagram, the curvature 
of the space surrounding the black hole is 
represented by the surface on which the 
waves propagate. The white ring locates 
the surface of the black hole, and the re- 
gions above and below represent the ex- 
terior and interior of the black hole, re- 
spectively. 


© 1989 David Hobill, Larry Smarr, David Bernstein, 
Donna Cox, and Ray Idaszak, NCSA, Univ. of lilinois 
at Urbana-Champaign. 
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Computational fluid dynamics. Com- 
putational astronomers rely on supercom- 
puting and visualization techniques to 
understand why jets from some galaxies 
flare dramatically. Magnetohydrodynam- 
ics code is used to solve equations that 
describe the fiow of a fluid or gas with 
magnetic fields using finite differences. 


-~ The above image is a visualization of a 


cosmic jet traveling at Mach 2.5 passing 
through a shock wave (located at the left 
of the image). The jet abruptly slows and 
breaks up into a broadened subsonic 


plume whose morphology, or shape, is 
Strikingly similar to that of a radio lobe of 
a wide-angle tailed galaxy. The morphol- 
ogy of the jet after impact is emphasized 
through the use of pseudocolor. This re- 
search has given astronomers important 
clues about why jets from some radio gal- 
axies flare into broad plumes while jets 
from others remain remarkably straight 
and narrow. 

© 1989 Michael Norman and Donna Cox of the NCSA, 


Univ. of Illinois at Urbana-Champaign, and Jack Bums 
and Martin Sulkanen of the Univ. of New Mexico. 
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Finite element analysis. Finite ele- 
ment dhalysis is used in this example to 
show the stress distribution in a beam at 
its maximum tip displacement in the third 
eigenmode. The results were computed 
using linear elastic elements and a 
jumped-mass approximation. 

The top image uses a conventional ap- 
proach of displaying the stress values on 
the outer surface of the deformed shape. 
The middie image uses a cutting plane to 
look at the stress values on a Cross sec- 
tion of the root of the beam. The bottom 
image shows a different view of the beam 
and uses an iso-contour stress surface to 
convey the three-dimensional nature of 
the stress concentration at the root of the 


beam. These images are still frames from 


fully animated and interactive models. 
They were computed and rendered ona 
Silicon Graphics 4D/120 GTX workstation 
using the SolidView program to perform 
real-time cutting and iso-contour surface 
generation. 


© 1989 James M. Winget, Silicon Graphics. 
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(Continued from p. 16) 


how visualization tools are helping re- 
searchers understand and steer computa- 
tions. Our examples fall into the following 
categories: 


¢ Molecular modeling, 

¢ Medical imaging, 

¢ Brain structure and function, 

° Mathematics, 

e Geosciences (meteorology), 

e Space exploration, 

¢ Astrophysics, 

¢ Computational fluid dynamics, and 
¢ Finite element analysis. 


Toolmakers’ 
short-term needs 


Commercial industry currently supports 
visualization hardware and software, as 
listed below. There is a pressing need to 
educate the scientific and engineering re- 
search communities about the available 
equipment. 


Software. Commercial visualization 
software exists in the following categories: 


Lines. The earliest software for graphics 
drew lines in three dimensions and pro- 
jected them onto a two-dimensional plane, 
offered viewing transformations for look- 
ing at the result, and offered transforma- 
tions (scale, rotate, and translate) for de- 
scribing the line objects. The theory and 
practice of drawing lines, expressed in 
homogeneous coordinates, and the control 
and display of lines using 4 x 4 matrices, 
represented a major development in com- 
puter graphics. 

A variety of current standards incorpo- 
rate these basic principles, and the CAD/ 
CAM industry has embraced this level of 
the art. It is cheap enough to put on every 
engineer's desk and fast enough for real- 
time interaction. 


Polygonal surfaces. The next level of 
software — surfaces represented by poly- 
gons — has only recently been built into 
hardware. Polygon filling, or tiling, is 
commonly available in hardware and soft- 
ware. Hidden surface removal is included, 
and antialiasing of polygon edges is some- 
times provided to remove distracting stair- 
steps, or jaggies. Light sources can be 
incorporated into the rendered image, but 
they are usually point sources at infinity 
emitting white light. 
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Patches. The next level of sophistica- 
tion represents surfaces as curved surface 
pieces called patches. This is still largely a 
software domain, although we expect hard- 


_ ware to appear soon. The most advanced 


software packages handle a variety of 
patch types. They also provide very so- 
phisticated lighting models with multiple- 


_ colored lights and distributed or point 


sources located either at infinity or in the 
scene. 

Antialiasing is assumed, and the pack- 
ages handle optical effects such as trans- 
parency, translucency, refraction, and re- 
flection. Research software provides even 
more features that produce greater realism, 
such as articulated motion blur, depth-of 
field, follow focus, constructive solid 
geometry, and radiosity. 

The software contains no practical limit 
on scene complexity (such as the number 
of allowable polygons), but computation 
of highly complex scenes on a supercom- 
puter can take anywhere from 0.5 to 1.5 
hours per frame. 


Image processing. Image processing 
software has followed a separate path over 
the last 15 years. The elaborate software 
packages now available provide functions 
such as convolution, Fourier transform, 
histogram, histogram equalization, edge 
detection, edge enhancement, noise reduc- 


tion, thresholding, segmentation, bicubic 


and biquadratic warping, and resampling. 

Many of these functions have been hard- 
wired into special boards. General-pur- 
pose processors have only recently be- 
come powerful enough to make software 
competitive with hardware while main- 
taining generality. Image computers can 
run both computer graphics and image 
processing software packages. 


Animation. In its broadest sense, anima- 
tion means movement. It frequently con- 
notes the complex motion of many objects, 
possibly articulated, moving simultane- 
ously, and interacting with one another. 
Animation is desirable for the visualiza- 
tion of dynamic, complex processes. Basic 
animation control routines should be part 
of any standard visualization tool kit. 


Glue. A class of software appreciated by 
visualization professionals but not neces- 
Sarily by scientists is the “glue” used to 
combine images generated or analyzed by 
the packages described above. For conven- 
ience, a user must have tools for picture 
composition, picture saving/restoring, 
fonts and text, resizing, rotation, moving, 


' copying, hand retouching (painting), color 


manipulation, etc. Together, these func- 
tions comprise a visualization environ- 
ment system, which is to visualization 
what an operating system is to general 
computing. 


Window systems. Windowing systems 
are commonplace in black-and-white bit 
graphics and are being extended to color 
graphics. Visualization software must in- 
corporate and remain consistent with win- 
dowing concepts. 


Volume visualization. Volume visuali- 
zation software is still rudimentary. Algo- 
rithms for rendering lines, curves, sur- 
faces, and volumes into volume memories 
are only now becoming available.>-* Hid- 
den volume removal is unknown, the com- 
positing of volumes is yet to be fully ad- 
dressed, 3D paint programs (sculpting 
programs) have yet to be written, and 
general utilities for arbitrary rotation and 
size change of volumes do not exist. In 
other words, there is much research to be 
done in this field. 


Hardware. The following are available 
commercial visualization hardware tools: 


Input devices. Current digital input 
devices include supercomputers, satel- 
lites, medical scanners, seismic recorders, 
and digitizing cameras. The rapidly in- 
creasing bandwidth of these devices em- 
phasizes the need for work in volume visu- 
alization. 

We expect continued improvement in 
the resolution and bandwidth of input 
devices. Supercomputers will get faster 
and the resolution of images from satellites 
will increase. Real-time video digitizers 
already exist. Monochrome digital digitiz- 
ers with 2,048 x 2,048-pixel resolution are 
becoming quite fast, although they do not 
yet operate at real-time speeds. Print-qual- 
ity input scanners are still quite expensive, 
but we expect the prices to fall as digital 
technology cheapens and competing scan- 
ning technologies mature. CCD (charge- 
coupled device) array input scanners will 
improve in resolution and become serious 
candidates for input devices in high-reso- 
lution work. 

Interactive input devices are continually 
improving. Common 2D devices include 
knobs, switches, pedals, mice, and tablets. 
Tablets are the most general and also reed 
the most improvement; they need higher 
resolution, higher speed, and more degrees 
of freedom. 
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Six-dimensional interactive devices are 
also available, providing the usual 3D 
positional information plus three degrees 
of orientation information (yaw, pitch, and 
roll). Higher-dimensional devices, such as 
the data glove, have begun to appear. These 
will improve to offer higher resolution, 
higher speed, and lower cost. 


Output devices. Raster displays of 2D. 


frame buffers have improved steadily to 
offer more colors, higher resolutions, and 
less flicker. A typical color raster display 
today offers a 1,280 x 1,024-pixel display 
at 60 frames per second and 24 bits of color 
per pixel (16 megacolors). 

High-definition television (HDTV) —a 
proposed standard that will offer larger, 
brighter, sharper pictures than currently 
available in video — will affect visualiza- 
tion. Also, video is moving toward an all- 
digital format, designated 4:2:2, to stan- 
dardize digital interconnections of diverse 
video products. 

Color raster displays will evolve toward 
2,048 x 2,048 pixels in the next several 
years. The displays themselves already 
exist in limited quantities, but the compu- 
tational bandwidth required to feed them is 
still lacking. Black-and-white 2D raster 
displays already have resolutions greater 
than 2,048 x 2,048 pixels with enough 
bandwidth to feed them. These displays 
will certainly reach even higher resolu- 
tions.in the next five years. 

Stereo displays are also beginning to 
appear commercially, and we confidently 
predict that these will improve in screen 
size, resolution, brightness, and availabil- 
ity. These displays will be quite helpful in 
volume visualization. 

Other output devices are similarly im- 
proving. HDTV will spur the development 
of compatible recorders. Film recorders 
will become cheaper as the technology 
becomes cheaper and the competition 
matures. Should stereo become a widely 
accepted mode of presentation for volume 
visualizations, then stereo film and video 
standards will have to be developed. 


Workstations. Fast vector machines are 
now common and have extensive use in 
such areas as CAD/CAM and real-time 3D 
design. Recently, they have improved to 
_ offer color vectors and perfect end-match- 
ing. Frame buffers have been added so that 
surface raster graphics can be combined 
with vector displays. 

Also, fast surface machines are about to 
arrive. They exist in simplified forms al- 
ready and in more advanced states as firm- 
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ware in special machines. Chips are now 
being built to speed up certain aspects of 
surface rendering, particularly the tiling of 
polygons. By 1990, full hardware support 
of surface graphics will be available, offer- 
ing rendering features such as texture 
mapping, bump mapping, antialiasing, 
reflections, transparency, and shadows. 

Vector machines will initially serve as 
powerful, real-time front ends to surface 
machines. Eventually, surface machines 
will be cheap and fast enough to permit 
scientists to do real-time design using sur- 
faces rather than lines. 

Among image processors, fast planar 
machines have existed for some time. 


These machines contain special boards for | 


certain aspects of image processing, such 
as fast Fourier transforms. Faster versions 
are becoming available that have wider 
processing capabilities and higher resolu- 
tions. In fact, the notion of a general-pur- 
pose image processor that can implement 
any image processing algorithm as a pro- 
gram is becoming common. 


Toolmakers’ 
long-term needs 


Raw computing power would be more 
effectively harnessed than it is today if 
calculations could be understood pictori- 
ally and their progress guided dynami- 
cally. Modern modes of computing involve 
interactive, extemporaneous generation of 
views from masses of data and exploration 
of model spaces by interactive steering of 
computations. 

A scientist’s ability to comprehend the 
results of his or her computations depends 
on the effectiveness of available tools. To 
increase that effectiveness, we need to 


e encourage the production of docu- 
mented, maintained, upward-compat- 
ible software and hardware; 

e motivate manufacturers to solve net- 
work bottleneck problems; 

* encourage universities to incorporate 
CS&E and visualization in computer 
science, engineering, and discipline- 
science curricula; and 

* guarantee the publication and dissemi- 
nation of research and results on a 
variety of media. 


Hardware, software, and systems. 
General visualization issues that need to be 
supported include: 


e Interactive steering of simulations and 
calculations 


¢ Workstation-driven use of supercom- 
puters 

¢ Graphics-oriented programming envi- 
ronments 

¢ Higher-dimensional visualization of 
scalar, vector, and tensor fields 

¢ Dynamic visualization of fields and 
flow 

¢ High-bandwidth picture networks and 
protocols 

e Massive data-set handling, notably for 
signal and image processing applica- 
tions 

° Vectorized and parallelized algo- 
rithms for graphics and image process- 
ing 

e Specialized architectures for graphics 
and image processing 

e A framework for international visuali- 
zation hardware and software stan- . 
dards 


Networking. The application of net- 
works to visualization, called televisuali- 
zation, encompasses much more than text 
transfer (such as electronic mail) and gate- 
way protocol decoding. It also involves 
image transfer, whichentails compression, _ 
decompression, rendering, recognizing, 
and interpreting. Televisualization re- 
quires a major enhancement over existing 
network capabilities in the following ar- 
eas: : 


Increased data rates. The sheer scale of 
graphics and imaging data sets challenges 
the current bandwidth and interactivity of 
networks. Networks handle screenfuls of 
textual information well; network nodes 
are simply gateways that neither add nor 
detract from the quality of the message. 
But a512 x 512-pixel image with 8 bits per 
pixel has approximately 100 times more 
inform&tion than a screen of text with 25 
rows and 80 characters per row. A 1,024 x 
1,024 x 1,024-voxel volume with 48 bits 
per voxel contains 16,000 times more in- 
formation than a 512 x 512-pixel image. 
Gigabit speeds are sufficient to pass vo- 
lumes of the current size of 256 x 256 x 
256 voxels with 4 bytes per voxel, but this 
rate will have to be extended within several 
years to 1-gigabyte/second channels. 


Compression/decompression algo- 
rithms. Compression improves the speed 
with which visual data is transmitted. 
Current schemes for. full-color image 
compression work well,’ but other forms 
of compression must be researched, and 
comprehensive protocols must be devel- 
oped for managing all these capabilities: 
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Table 4. The evolution of communica- 
tion tools. 


Number of 
years old 


Communications 
media 


Sight 
Speech 


Writing 

Print broadcasting 
Visual broadcasting 
Visualization 


¢ Transmit the procedures to create the 
images rather than the images them- 
selves. 

¢ Transmit endpoints of vector images. 

¢ Transmit polygonal, constructive 
solid geometry, or bicubic patches of 
surface models. 

¢ Transmit semantic descriptions of the 
objects. 


Value-added processing at nodes. 
Value-added nodes also speed up graphi- 
cal transmission. Computers process text 
and numbers in main memory, occasion- 
ally transmitting some of them to peripher- 
als. Images, however, often must be trans- 
ferred to special memories for rendering, 
3D imaging, or viewing. Each instance of 
transferring and processing an image aims 
to increase its visualization value to the 
scientist. The ability to process images at 
various nodes along a network embraces 
the central concept of distributed process- 
ing. 

-In distributed computing, transmission 
costs are balanced with local computing 
costs. It sometimes makes more sense to 
send model data over networks and then 
render or reconstruct the data at the 
scientist’s end. This presumes that there is 
appropriate equipment at both ends, that 
the various software modules are compat- 
ible with one another, and that the software 
can run on a variety of equipment types. 

A televisualization network for image 
passing between machines is analogous to 
the software paradigm of message passing 
between process layers. This type of net- 
working, combined with interaction, can- 
not be achieved using conventional For- 
tran subroutine calls. Significant software 
development and protocol standardization 
are necessary to bring televisualization to 
the discipline sciences. 


Interaction capabilities. Interactive vis- 
ual computing is a process whereby scien- 
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tists communicate with data by manipulat- 
ing its visual representation during proc- 
essing. The more sophisticated process of 
navigation allows scientists to dynami- 
cally modify, or steer, computations while 
they are occurring. This lets researchers 
change parameters, resolution, or repre- 
sentation, and then see the effects. 


Teaching CS&E and visualization. 
The principal barrier to growth in the 
CS&E market is the fact that corporate 
researchers and managers lack education 
and training in CS&E technologies and 
methodologies. Few industrial researchers 
know how to use distributed CS&E to do 
their work and, more importantly, they do 
not know how to think computationally 
and visually. Other roadblocks include the 
following: 


«The Association for Computing 
Machinery’s approved computer science 
curriculum lists computer graphics as 
merely one of many optional topics; image 
processing is not mentioned at all. 

¢ Engineering accreditors do not require 
computer graphics or image processing. 

e Many engineering school deans are 
unaware of the importance of visualization 
or cannot justify the hardware and soft- 
ware expense involved in teaching the 
subject. 

¢ The number of tenured faculty teach- 
ing computer graphics in American uni- 
versities is about the same today as 15 
years ago, and they are roughly the same 
people. 

¢ Scientists, while educated to read and 
write, are not taught to produce or commu- 
nicate with visuals. 


Publication and dissemination. Con- 
temporary scientific communications 
media are predominantly language-ori- 
ented. Printed media are coupled weakly, 
if at all, to the visual world of space-time. 
By contrast, half the human neocortex is 
devoted to processing visual information. 
In other words, current scientific commu- 
nication leaves out half — the right half — 
of the brain. An integral part of our visuali- 
zation task is to facilitate visual communi- 
cation from scientist to scientist, engineer 
to engineer, through visualization-com- 
patible media. | 

Publication and grants, and therefore 
tenure, rarely come to researchers whose 
productivity depends on or produces visu- 
alization results. Superiors evaluate schol- 
arly work by counting the number of jour- 
nal articles published; publications are 
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text, and visual media do not count. Fund- 
ing itself is based on the careful prepara- 
tion and evaluation of proposals, which are 
documents full of words and numbers. 

As scientists depend more and more on 
the electronic network than on the printed 
page, they will need new technologies to 
teach, document, and publish their work. 
Until scientists can build on each other’s 
work, productivity: will lag. Publishing 
(specifically textual materials) has always 
been a critical part of this building process, 
and it is one of the primary bottlenecks in 
CS&E’s progress. 

Reading and writing were only democ- 
ratized in the past 100 years. Today, they 
are the accepted communication tools for 
scientists and engineers. Table 4 shows 
that, in time, visualization will also be 
democratized and embraced by research- 
ers. 

Electronic media, such as videotapes, 
optical disks, and floppy disks, are now 
necessary for the publication and dissemi- 
nation of mathematical models, process- 
ing algorithms, computer programs, ex- 
perimental data, and scientific simula- 
tions. The reviewer and the reader need to 
test models, evaluate algorithms, and exe- 
cute programs themselves, interactively, 
without an author’s assistance. Similarly, 
scientific publication must extend to use 
visualization-compatible media. 


he use of visualization in scien- 

tific computing — in academia, 

government research laborato- 
ries, and industry — will help guarantee 


e US preeminence in science and tech- 
nology, 

¢ a well-educated pool of scientists and 
engineers with the quality and breadth 
of experience required to meet the 
changing needs of science and society, 
and 

¢ American industries that can success- . 
fully compete in the international eco- 
nomic arena. 


The information age has yet to deal 
with information transfer. Visualization 
technologies can help lead the way to bet- 
ter global understanding and communi- 
cation. 0 
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